237 research outputs found
Improved GelSight Tactile Sensor for Measuring Geometry and Slip
A GelSight sensor uses an elastomeric slab covered with a reflective membrane
to measure tactile signals. It measures the 3D geometry and contact force
information with high spacial resolution, and successfully helped many
challenging robot tasks. A previous sensor, based on a semi-specular membrane,
produces high resolution but with limited geometry accuracy. In this paper, we
describe a new design of GelSight for robot gripper, using a Lambertian
membrane and new illumination system, which gives greatly improved geometric
accuracy while retaining the compact size. We demonstrate its use in measuring
surface normals and reconstructing height maps using photometric stereo. We
also use it for the task of slip detection, using a combination of information
about relative motions on the membrane surface and the shear distortions. Using
a robotic arm and a set of 37 everyday objects with varied properties, we find
that the sensor can detect translational and rotational slip in general cases,
and can be used to improve the stability of the grasp.Comment: IEEE/RSJ International Conference on Intelligent Robots and System
Active Clothing Material Perception using Tactile Sensing and Deep Learning
Humans represent and discriminate the objects in the same category using
their properties, and an intelligent robot should be able to do the same. In
this paper, we build a robot system that can autonomously perceive the object
properties through touch. We work on the common object category of clothing.
The robot moves under the guidance of an external Kinect sensor, and squeezes
the clothes with a GelSight tactile sensor, then it recognizes the 11
properties of the clothing according to the tactile data. Those properties
include the physical properties, like thickness, fuzziness, softness and
durability, and semantic properties, like wearing season and preferred washing
methods. We collect a dataset of 153 varied pieces of clothes, and conduct 6616
robot exploring iterations on them. To extract the useful information from the
high-dimensional sensory output, we applied Convolutional Neural Networks (CNN)
on the tactile data for recognizing the clothing properties, and on the Kinect
depth images for selecting exploration locations. Experiments show that using
the trained neural networks, the robot can autonomously explore the unknown
clothes and learn their properties. This work proposes a new framework for
active tactile perception system with vision-touch system, and has potential to
enable robots to help humans with varied clothing related housework.Comment: ICRA 2018 accepte
Connecting Look and Feel: Associating the visual and tactile properties of physical materials
For machines to interact with the physical world, they must understand the
physical properties of objects and materials they encounter. We use fabrics as
an example of a deformable material with a rich set of mechanical properties. A
thin flexible fabric, when draped, tends to look different from a heavy stiff
fabric. It also feels different when touched. Using a collection of 118 fabric
sample, we captured color and depth images of draped fabrics along with tactile
data from a high resolution touch sensor. We then sought to associate the
information from vision and touch by jointly training CNNs across the three
modalities. Through the CNN, each input, regardless of the modality, generates
an embedding vector that records the fabric's physical property. By comparing
the embeddings, our system is able to look at a fabric image and predict how it
will feel, and vice versa. We also show that a system jointly trained on vision
and touch data can outperform a similar system trained only on visual data when
tested purely with visual inputs
Slow and Smooth: A Bayesian Theory for the Combination of Local Motion Signals in Human Vision
In order to estimate the motion of an object, the visual system needs to combine multiple local measurements, each of which carries some degree of ambiguity. We present a model of motion perception whereby measurements from different image regions are combined according to a Bayesian estimator --- the estimated motion maximizes the posterior probability assuming a prior favoring slow and smooth velocities. In reviewing a large number of previously published phenomena we find that the Bayesian estimator predicts a wide range of psychophysical results. This suggests that the seemingly complex set of illusions arise from a single computational strategy that is optimal under reasonable assumptions
Separating Reflections from Images Using Independent Components Analysis
The image of an object can vary dramatically depending on lighting, specularities/reflections and shadows. It is often advantageous to separate these incidental variations from the intrinsic aspects of an image. Along these lines this paper describes a method for photographing objects behind glass and digitally removing the reflections off the glass leaving the image of the objects behind the glass intact. We describe the details of this method which employs simple optical techniques and independent components analysis (ICA) and show its efficacy with several examples
GelSight Svelte Hand: A Three-finger, Two-DoF, Tactile-rich, Low-cost Robot Hand for Dexterous Manipulation
This paper presents GelSight Svelte Hand, a novel 3-finger 2-DoF tactile
robotic hand that is capable of performing precision grasps, power grasps, and
intermediate grasps. Rich tactile signals are obtained from one camera on each
finger, with an extended sensing area similar to the full length of a human
finger. Each finger of GelSight Svelte Hand is supported by a semi-rigid
endoskeleton and covered with soft silicone materials, which provide both
rigidity and compliance. We describe the design, fabrication, functionalities,
and tactile sensing capability of GelSight Svelte Hand in this paper. More
information is available on our website:
\url{https://gelsight-svelte.alanz.info}.Comment: Submitted and accepted to IROS 2023 workshop on Visuo-Tactile
Perception, Learning, Control for Manipulation and HRI (IROS RoboTac 2023
GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing
Camera-based tactile sensing is a low-cost, popular approach to obtain highly
detailed contact geometry information. However, most existing camera-based
tactile sensors are fingertip sensors, and longer fingers often require
extraneous elements to obtain an extended sensing area similar to the full
length of a human finger. Moreover, existing methods to estimate proprioceptive
information such as total forces and torques applied on the finger from
camera-based tactile sensors are not effective when the contact geometry is
complex. We introduce GelSight Svelte, a curved, human finger-sized,
single-camera tactile sensor that is capable of both tactile and proprioceptive
sensing over a large area. GelSight Svelte uses curved mirrors to achieve the
desired shape and sensing coverage. Proprioceptive information, such as the
total bending and twisting torques applied on the finger, is reflected as
deformations on the flexible backbone of GelSight Svelte, which are also
captured by the camera. We train a convolutional neural network to estimate the
bending and twisting torques from the captured images. We conduct gel
deformation experiments at various locations of the finger to evaluate the
tactile sensing capability and proprioceptive sensing accuracy. To demonstrate
the capability and potential uses of GelSight Svelte, we conduct an object
holding task with three different grasping modes that utilize different areas
of the finger. More information is available on our website:
https://gelsight-svelte.alanz.infoComment: Submitted and accepted to 2023 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS 2023
GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger
This work describes the development of a high-resolution tactile-sensing
finger for robot grasping. This finger, inspired by previous GelSight sensing
techniques, features an integration that is slimmer, more robust, and with more
homogeneous output than previous vision-based tactile sensors. To achieve a
compact integration, we redesign the optical path from illumination source to
camera by combining light guides and an arrangement of mirror reflections. We
parameterize the optical path with geometric design variables and describe the
tradeoffs between the finger thickness, the depth of field of the camera, and
the size of the tactile sensing area. The sensor sustains the wear from
continuous use -- and abuse -- in grasping tasks by combining tougher materials
for the compliant soft gel, a textured fabric skin, a structurally rigid body,
and a calibration process that maintains homogeneous illumination and contrast
of the tactile images during use. Finally, we evaluate the sensor's durability
along four metrics that track the signal quality during more than 3000 grasping
experiments.Comment: RA-L Pre-print. 8 page
GelSight360: An Omnidirectional Camera-Based Tactile Sensor for Dexterous Robotic Manipulation
Camera-based tactile sensors have shown great promise in enhancing a robot's
ability to perform a variety of dexterous manipulation tasks. Advantages of
their use can be attributed to the high resolution tactile data and 3D depth
map reconstructions they can provide. Unfortunately, many of these tactile
sensors use either a flat sensing surface, sense on only one side of the
sensor's body, or have a bulky form-factor, making it difficult to integrate
the sensors with a variety of robotic grippers. Of the camera-based sensors
that do have all-around, curved sensing surfaces, many cannot provide 3D depth
maps; those that do often require optical designs specified to a particular
sensor geometry. In this work, we introduce GelSight360, a fingertip-like,
omnidirectional, camera-based tactile sensor capable of producing depth maps of
objects deforming the sensor's surface. In addition, we introduce a novel
cross-LED lighting scheme that can be implemented in different all-around
sensor geometries and sizes, allowing the sensor to easily be reconfigured and
attached to different grippers of varying DOFs. With this work, we enable
roboticists to quickly and easily customize high resolution tactile sensors to
fit their robotic system's needs
- …